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ABSTRACT 



This symposium on university programs consists of three 
presentations. "Institutional and Curricular Characteristics of Leading 
Graduate Human Resource Development (HRD) Programs in the United States" (K. 
Peter Kuchinke) reports a study on institutional arrangements, student 
enrollments, and core curriculum content and found a large degree of 
heterogeneity among program names, departmental affiliations, and 
specializations; declining student enrollment; increased part-time course 
taking; and disparity between course offerings and much current writing in 
the field. "Assessment of a Graduate Program in HRD: Perceptions of Key 
Stakeholders" (Julie A. Furst-Bowe, Joseph Benkowski) discusses these 



findings: overlap among courses; difficulty in providing administrative and 
student services to students; need for additional staff; lack of quality 
classrooms and instructional facilities; and quality and level of research 
and scholarship among faculty and students. "Predicting Academic Performance 
in Management Education: An Empirical Investigation of Master of Business 
Administration (MBA) Success" (Baiyin Yang, Xiaoping Rosa Lu) suggests that 
undergraduate grade point average was the most important predictor for the 
graduate academic performance, followed by Graduate Management Admission Test 
(GMAT) quantitative and GMAT verbal, while language made little contribution. 
All three papers include substantial bibliographies. (YLB) 
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Institutional and Curricular Characteristics of Leading Graduate HRD Programs in the 
U.S. 



K. Peter Kuchinke 
University of Illinois 

Focusing on three areas: Institutional arrangements, student enrollments, and core curriculum 
content, the study profiles 55 leading HRD programs. Findings include a large degree of 
heterogeneity among program names, departmental affiliations, and specializations. Compared to 
data from 1991, student enrollment has declined substantially at the master's level while part-time 
course taking has increased. The analysis of the core curriculum at these institutions showed a 
disparity between course offerings and much current writing in the field. 

Keywords: HRD programs, HRD curriculum 

Responding to the increasing demand for employee skills, expertise, and performance in rapidly changing economic 
and social environments, many universities have implemented academic programs to educate and train HRD 
practitioners over the past 15 years. Today, these programs are firmly established in professional schools and are in 
high demand among students from a variety of academic backgrounds. In U.S. Schools of Education, HRD 
enrollments are among the fastest growing and the “the training of ... HRD practitioners is now the ‘bread and 
butter’ activity” (Gray, 1997, p. 80). 

Little, however, is known about the departments and programs that educate and train future HRD 
practitioners. Increased knowledge about the characteristics of HRD programs is important because their roles are 
likely to increase in importance as the profession grows and matures. With increased significance and professional 
status of a field, as Abbot (1988) observed, formal education and training and the attainment of university degrees 
and professional credentials take on increasingly important roles in career preparation and advancement. The 
systematic study of HRD academic programs and departments, thus, is an important but largely overlooked area of 
scholarship in the field. The purpose of this study was to respond to the need to broaden the knowledge base related 
to the characteristics of HRD academic programs in the US by answering three questions: 

• What are the institutional characteristics of HRD programs? 

• How many students are enrolled in different HRD degree and certification tracks? 
o What is the core body of knowledge taught in HRD programs? 

Answers to these questions can serve several purposes: First, they build a foundation for describing the 
status of the field in an area where empirical research is lacking — many authors describe what HRD education and 
training should be but far fewer publications address the empirical reality of what is. Second, answers to these 
questions allow researchers to track the development of HRD education over time and assess the degree to which 
educational institutions adapt to the changing needs of their various stakeholders. Third, these answers lay the 
foundation for comparative research of systems of human resource education at the national level (the author is 
currently completing a similar study in the UK). Especially for a young field like HRD, institutional research is 
important and has been conducted fruitfully in other fields (for example: adult education: The University of Georgia, 
1999; vocational education: Lynch, 1998; management education: Segev, Raveh, & Farjoun, 1999; and international 
MBA programs: Kaynack & Schermerhom, Jr., 1999). 

Review of Related Literature 

Problems abound when trying to define the universe of HRD programs in the U.S. In the absence of a central 
accrediting body at the program level and institution-level professional organizations, little is known about the total 
number of programs in this country. This is in contrast to the United Kingdom where academic preparation for HRD 
is conducted by well-defined group of about 30 programs accredited by a national body that is also responsible for 
skill standards and professional certification. In the U.S., however, higher education is decentralized to a very high 
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degree and thus, reliable data on the number of HRD programs are not available. Of the 3,595 public and private 
institutions of higher learning catalogued by the Carnegie Foundation for the Advancement of Teaching (1994) 
some 1,400 offer 4-year (baccalaureate) degrees or higher. The most extensive listing of HRD programs was 
compiled by the American Society for Training and Development (ASTD, 1996) and lists more than 250 degree and 
certificate programs located in HRD and related programs in a variety of fields. 

The ASTD directory presents the most comprehensive listing of programs available but is based on 
voluntary information provided by the units in response to a mail survey and is, thus, likely not to be exhaustive. 
Further, ASTD has decided not to update the directory, citing the reason that HRD programs are easy to find on the 
World-Wide Web and in college handbooks published annually in the U.S. with profiles and rankings of degree 
programs (ASTD, personal communication, November 1, 1999). 

The most comprehensive survey of HRD academic programs was conducted by Gaudet and Vincent in 
1993. Based on 1991 data, the authors surveyed universities listed in the 1990 ASTD directory and reported on 122 
responding institutions. HRD was the program name used most frequently (n=42), followed by Adult Education 
(n=22), Instructional Technology (n=12), and Training and Development (N=7). The average program was reported 
to be 14.5 years old in 1991 and the majority of programs were implemented in the 1980s (Gaudet & Vincent, 
1993). Although this study is well over 10 years old, it is surprising to see the total number of HRD programs listed 
in the 1990 and 1996 ASTD directories being quite similar (218 in 1990, 250 in 1997). While it is not possible to 
determine the overlap of institutions in the two directories, the similar size of the two suggests that the total number 
of institutions offering HRD degrees increased by less than 15% in the 1990s. 

Enrollments 

Accurate enrollment data of HRD programs are also difficult to obtain. The only available study is, again, 
Gaudet and Vincent's (1993) who reported — among the 113 HRD programs surveyed — 41 bachelor's, 108 masters, 
and 45 doctoral programs with a total student enrollment about 12,200 full-time and 19,300 part-time students. 
Thus, virtually all programs offered Master's degrees, while some also had undergraduate programs, doctoral 
programs, or both. Among full-time students about 65% were working towards a Bachelor's, 29% towards a 
Master's, and 6% towards a doctoral degree. For part-time students the percentages were 49%, 41%, and 10% 
respectively. The average program graduated 54 Bachelor's, 20 Master's, and 4 Ph.D. students in the 1990/1991 
school year resulting in an annual labor supply of about 2,200 Bachelor's, 2,160 Master's, and 180 doctoral degree 
holders. 

Most studies on the curriculum of HRD academic programs use McLagan's (1983; 1989; 1996) research as 
a point of reference. While other competency frameworks exist, particularly for trainers (for example; Fulkert, 
1997, Mager, 1996; Marquardt & Engel, 1993), MacLagan's work has been "adopted as a definitive model of 
competencies by the American Society for Training and Development [and] appears to continue to provide the basis 
for preparation of HRD professionals" (Dare & Leach, 1999, p. 2). In 1991, for example, Leach surveyed among 20 
departments belonging to the University Council of Vocational Education (now named: University Council for 
Workforce and Human Resource Education). Faculty at those departments, all located in Ph.D. granting institutions, 
considered 21 of the 25 competencies for trainers described by McLagan as important for graduates of their 
programs, although more than one-half were deemed not be adequately covered in the curriculum of the respective 
institution (Leach, 1993). A follow-up study (Dare & Leach, 1999) identified increased significance of research 
skills, electronic-systems skills, and visioning, but a decreased importance of competency identification and 
objective preparation skills. As in the original study, several competencies were judged not to be adequately covered 
in the curriculum, including career development theories and techniques, budget and resource management skills, 
organization development theories and techniques, and coaching and negotiation skills. 

Baylen, Bailey, and Samardzija (1996) investigated HRD curricula at four research universities using, 
again, McLagan's framework and found that the six HRD roles most thoroughly addressed in HRD courses were, in 
rank order: Instructor, Program Designer, Manager (Training and Development/HRD), Needs Analyst, 
Organizational Change Agent, and Evaluator. Of the 35 HRD courses analyzed by Baylen et al. (1996), only two 
addressed the emerging HRD roles of budget/cost analyst and strategic planner, four addressed the role of Manager 
of Change, and six the role of Performance Technologist. Thus, the study, although restricted to only four programs, 
shows a relatively conservative picture of HRD curricula primarily focused on training roles with only minor 
coverage of broader professional roles and responsibilities. 
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Methodology 



The purpose of this study was to investigate the institutional and curricular characteristics of U.S. HRD programs by 
focusing on institutional arrangements, enrollment, and curricula. Since previous research (Gaudet & Vincent, 1993) 
had shown that most HRD programs offered graduate education, the population for this study was defined as those 
programs that offer master's, doctoral, or graduate-level certificate programs, excluding undergraduate and 
undergraduate-level certificate programs. Beginning with the 120 graduate programs listed in last available ASTD 
(1996) directory, the researcher selected 67 graduate HRD programs with one or more faculty members having 
been, in any of the previous three years, members of the Academy of Human Resource Development (AHRD), the 
primary professional association. The rationale for this selection was those institutions with faculty active in the 
AHRD would best represent comprehensive programs focused on the entire field of HRD rather than those programs 
where HRD is a minor sequence of courses or a specialization within another field of study. The review of AHRD 
rosters showed that faculty membership was relatively stable between 1997 and 1999, with many members who also 
held office in the AHRD and actively participated as presenters in the annual conference and thus could be assumed 
to present the leading edge of the profession. Program administrators of the 67 programs were contacted and invited 
to participate in the study, and 55 (82%) agreed to do so. A document review of written program information, 
brochures, and syllabi as well as material available on the programs' web sites was conducted, and information 
recorded and coded. In a second step, this information was sent to the program administrators for validation, 
correction, and missing information. 41 administrators replied for a validation rate of 75 percent. Because the 
corrections were, in most cases, minor, the researcher decided to include the 14 programs where validation, despite 
several attempts, could not be obtained after a second thorough review of available materials and documents was 
conducted for these programs. 

Results 

The sample of 55 HRD graduate programs, listed in Appendix A, contained 44 Ph.D. granting and 11 master's 
programs. The sample included all but one of the 18 institutions that were also members of the University Council 
for Workforce and Human Resource Education, a professional organization with institutional membership of leading 
research universities. 21 of the universities in the sample were also listed in the Peterson guide of graduate programs 
as HRD programs, while 30 were listed in the same guide as vocational/technical graduate programs, with some 
programs being listed in both categories. Using the Carnegie classification of institutions of higher education, the 
sample consisted of programs located in the following categories: Research I: 27, Research II: 6, Doctoral I: 7, 
Doctoral II: 4, Masters 1:11. 

Institutional Affiliation 

To describe the institutional affiliation of the sample, the following data elements were coded and 
tabulated: program name, departmental and college/school affiliation, HRD specializations, and related programs 
offered in the same department or unit. Many program names showed multiple program emphases (for example: 
Adult instruction and performance technology, or Workforce education and lifelong learning). The majority of 
programs (31 of 55), however, carried the title HRD, either alone or in conjunction with related field (for example: 
Adult Learning and HRD). The second most frequent program category was Instructional/Performance Technology 
(9 programs), followed by Adult education and related fields (8 programs), Vocational/Technical and workforce 
education (5 programs), Training and Development, and Adult/Organizational Learning with 4 programs each. 

The 55 programs were located in departments or larger units that were very heterogeneous in name, ranging 
from Departments of Public Administration and Urban Studies, to Psychology, Leadership and Organization, 
Management, Human Services, Counseling, Curriculum and Instruction, and many others. In fact, there were no two 
departments with the same name and only a small number with similar names (those indicating emphases on 
counseling, educational leadership, and adult education. 

The large majority of HRD programs (42 of 55) were located in colleges and schools of education, some of 
which were also identified as: education and psychology, health and education, education and human development, 
teacher's college, and so on. Two programs were in schools of business administration, one of which was a joint 
program between business and education. One program was affiliated with the human resource education and 
training center at a school of labor and industrial relations. 

About one-half of the programs (26 of 55) also listed specializations within the HRD programs. Perhaps not 
surprising given the diversity of departmental affiliations, no clear categories emerged from the listing of 58 
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specializations. Seven programs offered specializations focused on adult learning, including adult development and 
adult literacy, six programs on instruction design, three on performance technology, and another three on 
international HRD and international education respectively. Two program each offered specializations in 
organization development, information technology, educational leadership, and assessment/evaluation. The 
remaining specialization covered a broad area, ranging from conflict management, counseling, distance education, 
and facilitation to gerontology, public policy, the not-for-profit sector, and organizational learning. 

Asked for degree programs related to HRD that were offered in the same department or academic unit, 31 
programs listed some 57 related programs. These clustered around vocational/technical education and 
specializations within it, such as marketing, family, industrial, and business education (17 programs offering degree 
programs in these areas); adult education (seven programs); curriculum and instruction (five programs), and 
miscellaneous others, including health management, public administration, counselor education, and social and 
organizational psychology. 

Seventeen of the 55 programs also offered graduate-level certificate programs in HRD and related areas. 
These ranged from training and development, performance technology, web-based training, and instructional 
technology to vocational education and teacher licensure options. 

Enrollment 

Forty-two programs reported enrollment data, and these were broken down by full-time and part-time for 
masters and doctoral levels and non-degree certificate options. Three programs did not have separate counts 
available for part-time and full-time students but provided combined enrollment numbers. Several program 
administrators indicated that exact enrollment counts were not available and provided estimates instead. Table 1 
shows the enrollment data for the Fall 1999 semester. 

Table 1. Student Enrollment in HRD Programs in Fall Semester, 1999 (N=42) 





Master's (n=42) 


Doctoral (n=28) 


Certificate (n=13) 


Total (n=42) 


Overall 


3903 


1422 


462 


5787 


Average 


92.93 


50.79 


35.54 


137.79 


Median 


78 


29 


22 


123 


Mode 


150 


6 


50 


65 


s.d. 


66.52 


49.84 


36.30 


101.49 


Min/Max 


7/250 


4/245 


5/137 


4/403 


Percent part-time 


80 


77 


85 


80 



As the table shows, overall enrollment in graduate degree and certificate programs for the Fall of 1999 was 
close to 5,800 students, with an average enrollment in master’s, doctoral, and certificate programs of 93, 51, and 36 
students respectively. There was a wide range in enrollment counts as shown by the minimum/maximum numbers. 
The vast majority of HRD students attended on a part-time basis, and this was true for all three program categories. 

Curricula 

Tallying and comparing curricula across programs and institutions is difficult because of the wide variation 
in course titles and the fact that course names often reveal very little about actual course content. To address these 
difficulties, the researcher coded content areas covered in course rather than course titles and further focused on the 
core or required curriculum, ignoring elective courses. This information was obtained by analyzing course syllabi 
and through validation by the administrators. Where no required core curriculum existed, administrators were asked 
to indicate those content areas that the average student would learn over his or her course of study. From an initial 
review an initial listing of content areas was developed using 37 separate content areas that appeared to cover the 
content of the courses reviewed. This list was then sent to the program administrators with the request to validate the 
information, add content areas that were overlooked, and add areas not on the list. This resulted in a final list of 34 
areas that appeared to circumscribe the universe of content covered in the core or required curriculum of the 55 
HRD programs. The list excluded statistics and methods courses that were presumed to part of all doctoral students' 
preparation, and focused on HRD content areas only. 
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While other researchers used cluster analytic techniques to map like programs (for example: Segev, Raveh, 
& Farjoun, 1999), simple frequency distributions were calculated in this study because of the large number of 
programs and content areas and the focus on the study on describing curriculum rather than identifying clusters of 
programs. The 55 programs identified a total of 981 content areas, with the average program covering about one-half 
of the areas identified. After deleting two content areas that were judged to be too vague to add value to the analysis 
(HRD Trends and Issues, offered by 84% of the programs, and HRD Principles and Theories, offered by 53%), and 
merging one (adult learning theories and general learning theories) the final list consisted of 31 areas that were 
ranked in order of frequency and displayed in Table 2. 

Table 2. Content Areas Covered in Core/Required Curricula of Graduate HRD Programs (N=55) 



Rank 


Content Area 


Frequency 


Percent 


1 


Instructional Design 


43 


78% 


2 


Instructional Delivery 


41 


75% 


3 


Evaluation 


41 


75% 


4 


Adult Learning Theories 


36 


73% 


5 


Needs/Performance Analysis 


35 


65% 


6 


History and Philosophy of HRD 


35 


64% 


7 


Instructional Technology 


33 


60% 


8 


Organization Development 


33 


60% 


9 


HRD Consulting 


32 


58% 


10 


Management of HRD 


30 


55% 


11 


Organization Theory/Behavior 


30 


55% 


12 


Organizational Leaming/Leaming Organization 


29 


53% 


13 


Computer Applications in HRD 


28 


51% 


14 


Principles of Business/Industry/Mgmt 


27 


49% 


15 


Teams/Group Dynamics 


26 


47% 


16 


Change Management 


25 


45% 


17 


Diversity/Multicultural HRD 


24 


44% 


18 


Instructional Media 


23 


42% 


19 


Distance Learning 


23 


42% 


20 


Career Development 


23 


42% 


21 


Strategic HRD 


23 


42% 


22 


Psychological Dimensions in HRD 


22 


40% 


23 


Facilitation 


22 


40% 


24 


Communication in HRD 


20 


36% 


25 


Organization Analysis 


20 


36% 


26 


Leadership/Management Development 


20 


36% 


27 


International HRD 


19 


35% 


28 


Action Leaming/Research 


17 


31% 


29 


Economic Dimensions of HRD 


15 


27% 


30 


HRD/Educational Policy Studies 


12 


22% 


31 


Quality Management 


11 


20% 



As the table shows, HRD graduate program curricula offer coverage of a variety of content areas but with 
few outliers. Even the least frequently addressed area, quality management, is still offered by one in five programs. 
Instructional design and delivery lead the list with evaluation being the third most frequently addressed topic. These 
three areas were covered in more than three-fourth of all programs. More than one-half (but fewer than three- 
quarter) of all programs taught organization development, organizational leaming/leaming organization, consulting, 
and learning theory related topics. Given the increased attention of strategic and economic issues in the HRD 
literature, surprisingly few programs addressed these topics, and this was also true for diversity and international 
HRD. Quality management and public/educational policy related topics trailed the list with fewer than one-fourth of 
programs addressing these topics in their core curriculum. 




20-1 



Conclusion 



HRD graduate programs play an important role in the U.S. system of human resource development, educating and 
training future practitioners, researchers, and instructors. Little systematic knowledge, however, exists about these 
programs, and this gave rise to this study. While the universe of academic programs cannot be defined with 
certainty, the 55 graduate programs in this study are assumed to represent a major portion of the nation's leading 
institutions offering advanced education and training in HRD. The purpose of the study was to gain insight into three 
areas: institutional characteristics of those programs, student enrollments, and the HRD curriculum. 

Related the first, the study found that only one-half of programs actually carried the name HRD in their 
titles, although, based on self-identification, listing in professional directories, and active membership in the primary 
professional association, all were actively engaged in educating and training HRD master’s and doctoral students. 
Further, the 55 programs were located in departments and administrative units with a large array of different names, 
most of which gave little indication that they might be home to an HRD department or program of study. This 
finding might give the profession cause for concern because it calls into question its academic identity and might 
obscure the value associated with the academic provenance of its graduates. HRD differs from most other 
professions where academic departments are clearly labeled and transparent to the public, employers, and 
prospective students. Doctors, lawyers, teachers, and accountants, for example, have well-defined educational paths, 
and these are clearly recognized and recognizable. As HRD develops as a field, the preparatory paths of future 
practitioners might need to be explained more clearly, in particular with the majority of programs located in schools 
of education that are, in the public mind, first and foremost associated with public school teaching. Conceptual, 
definitional, and marketing work on part of the profession and individual programs is called for to argue and 
demonstrate the value and significance of HRD programs located in academic departments so seemingly 
disconnected from the world of business and industry. The value of even fields as closely related to HRD as 
vocational or adult education might not be immediately obvious to the business community, employers, or potential 
future HRD practitioners and thus needs to be explored and explained by the profession in order to broaden its 
recognition as a valued member of the business and organizational communities. 

The second research question centered on enrollments and points to two major findings: First, the size of 
student enrollment in graduate programs appears to have grown compared to the findings of Gaudet and Vincent's 
(1993) study of HRD programs in 1991. While it was not possible to determine the identity of the sample of older 
study, there are indications that substantial overlap exists: both seemed to have drawn from the same universe using 
the ASTD directories. While the overall number of programs increased by about 15 percent between 1991 and 1996, 
the 1999 average master's level enrollment decreased by almost one-half (172 in 1991 vs. 93 in 1999) while the 
average student enrollment at the doctoral level remained essentially the same (45 in 1991 vs. 51 in 1999). Whether 
this indicates an overall decrease in interest in the field, the availability of work opportunities in a tight labor market, 
or a broadening of the supplier base of HRD programs is open to further research. It does call into question, 
however, the popular wisdom that the HRD field is expanding rapidly because of the increased demand for HRD 
services and provisions. At least for the population in this study, overall enrollment decreased. 

A second insight related to student enrollment was the fact that eight out of ten students attended part-time, 
representing an increase of 15 percent for master's and 10 percent for doctoral students. Whether this is due to the 
increased availability of jobs for students, the fact that the student base is shifting towards mid-career practitioners, 
or other reasons is, again, open to further investigation. It should also be noted that part-time enrollment among 
doctoral students is rare among professional schools, although more prevalent in schools of education than in other 
units. The impact of part-time course taking on academic achievement and professional effectiveness also awaits 
further investigation. 

When looking at the HRD curriculum, it becomes apparent that while much of the conceptual writing in the 
field, stressing organizational learning, strategic HRD, international issues, workforce diversity, change 
management, distance learning, and the economic impact of HRD has not yet been translated universally into core 
curriculum content. Fewer than one-half of these programs that can, after all, be considered among the leading edge 
degree programs in the country, have included these content areas into their curriculum, and this might imply that 
not all graduates are as well prepared as they should be. Since these programs can also be assumed to educate a 
sizeable portion of future HRD faculty, one might ask whether future faculty members are as prepared as they need 
to be given the new developments in the field. It appears that the majority of graduate programs remain focused on 
rather conventional training and development content areas but are, perhaps, not consistently up to date with leading 
developments in the field. In this respect, little progress seems to have been made since Leach’s (1991) study that 
found more than one-half of HRD competencies were not adequately addressed by academic programs. 
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This study attempted to begin a systematic investigation into academic HRD programs, and, clearly, raised 
at least as many questions as it was able to answer. A few examples of the more pressing research needs are the 
following: How can the universe of HRD academic preparation be more accurately defined? What is the educational 
background of HRD practitioners and what percentage do get their education in HRD programs? What is the impact 
on professional effectiveness of different academic preparations? How much carry-over exists between academia 
and professional practice? How can supply and demand characteristics for HRD practice be more accurately 
identified? 

Lastly, several limitations require mentioning that should be kept in mind when making sense of these 
findings. First, this article does not represent the universe of HRD programs but a purposefully selected volunteer 
sample. Thus, the findings do not generalize beyond the 55 programs. Second, the comparisons to the earlier study 
by Gaudent and Vincent (1993) are subject to distortions, any differences, thus, might be the results of different 
populations. Finally, curriculum information provided by the programs is likely subject to differences in 
interpretation and likely over reporting; reported differences in content area coverage, thus, might be explained with 
real differences or with different interpretations of the content area labels. Finally, no attempt was made to quantify 
how much of a content area was covered, merely whether or not it was addressed. The survey also asked for core or 
required subject matter only, and this might have been interpreted differently by the respondents. 

The study of academic HRD programs is in its infancy but has the potential to contribute to a clearer, more 
consistent, and reasoned definition of the field and the knowledge, skills, and competencies for effective HRD 
practice. 

References 

Abbott, A. (1988J The system of profession s: An essay on the division of expert labor . Chicago: University 
of Chicago. 

Achtenhagen, F., & Grubb, W. N. (1999). Vocational and occupational education: Pedagogical complexity, 
institutional diversity. In V. Richardson (Ed.), Handbook of research on teaching (4 th cd ), Washington, DC: 
American Education Research Association. 

ASTD (1996, November). Academic directory; Programs in human resource development 1997. 
Alexandria, VA: American Society for Training and Development. 

Bureau of Labor Statistics, 1998. Occupational outlook handbook. Washington, DC: author. 

Baylen, D.M., Bailey, M. L., & Samardzija, M. (1996). Revisiting perceptions of HRD roles: Implications 
for HRD curricula. In E. Holton, III (Ed.), Academy of Human Resource Development 1996 Conference 
Proceedings , section 33-1. Baton Rouge, LA: Academy of Human Resource Development. 

Carnegie Foundation for the Advancement of Teaching (1994). The Carnegie classification of institutions 
oCbigheL education. Washington. DC: author. 

Clark, B. R. (1983). The higher education system: Academic organization in cross-national perspective. 
Los Angeles: University of California. 

Clark, B. R. (Ed.). The school and the university: An international perspective. Berkely, CA: University of 
California. 

Copa, G. H. (1992). A framework for the subject matter of vocational education. Berkeley: National Center 
for Research in Vocational Education, University of California, Report MDS-095. 

Copa, G. H., & Tebbenhoff, E. (1990). Subject matter of vocational education: In pursuit of foundations. 
Berkeley: National Center for Research in Vocational Education, University of California, Report MDS-094. 

Dare, D. E., & Leach, J. A. (1999). Preparing tomorrow's HRD professionals: Perceived relevance of the 
1989 competency model. Journal of Vocational and Technical J£du cation* 15 (2). 5-18. 

Fulkert, R. F. (1997). Competencies required to be a trainer: A delphi study. Doctoral dissertation, The 
University of Toledo. Dissertation AbstractsJniemationaL 58-04A. 

Garavan, T. N., Heraty, N., & Bamicle, B. (1999). Human resource development literature: Current issues, 
priorities, and dilemmas. Journal of European Industrial Traininfi^23 (4/5). 169-179. 

Gaudet, C., & Vincent, A. (1993, Summer). Characteristics of training and human resource development 
degree programs in the United States. The Delta Pi Epsilon Journal. 35 (3). 138-160. 

Gray, K. (1997). Seeking the ‘tie that binds’: Integrating training & development/human resource 
development and teacher preparation. Journal of Industrial Teacher Education. 34 (4). 80-85. 

Kaynack, E., & Schermerhom, J. R., Jr. (1999). Teaching and program variations in international business: 
Past, present, and future. Journal of Teaching in International Business. 10 (3/4). 1-10. 

Kuchinke, K. P. (in press). HRD university education: An international research agenda. Human Resource 
Development International. 




20-1 



Leach, J. A. (1993). Preparing tomorrow's business and industry trainers. Journal of Vocational and 
Technical Education. 9 (2). 4-16. 

Lynch, R. L. (1991). A national database on vocational teacher educaliorL Berkeley: National Center for 
Research in Vocational Education, University of California, 

Lynch, R. L. (1998). Vocational teacher education in U.S. colleges and universities, and its responsiveness 
to the Carl. D. Perkins Vocational and Applied Technology Education Act of 1990. In A. Gamoran (Ed.), The 
quality of vocational education: Background papersJrom the 1994 national-assessment of vocational education (pp. 
5-42). Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement. 

Lynch, R. L., & Griggs, M. B. (1989). Vocational teacher education: A context for the future. Berkeley: 
National Center for Research in Vocational Education, University of California. 

Mager, R. F. (1996, June) Morphing into a 21 st century trainer. Training. 33 (6). 65-74. 

Marquardt, M. J., & Engel, D. W. (1993, May). HRD competencies for a shrinking world. Training and 
Development. 47 (5). 59-60, 62-65. 

McLagan, P. A. (1989). Models for HRD practice . Alexandria, VA: American Society for Training and 
Development. 

McLagan, P. A. (1996) Great ideas revisited: Creating the future of HRD. Training and Development. 
5Q(1), 60-65. 

McLagan, P. A(Ed.). (1983 ) Models for excellence: The conclusions and recommendations of the ASXD 
training and development competency study. Washington, DC: American Society for Training and Development. 

Roberts, H.V. (1990). The quality revolution and the business school response. Chicago School of 
Business, Selected Paper Series. 

Rynes, S. L. and Trank, C. Q. (1999). Behavioral science in the business school curriculum: Teaching in a 
changing institutional environment. Academy of Management Review. 24 (4): 808-824. 

Segev, E., Raveh, A., & Farjoun, M. (1999). Conceptual maps of leading MBA programs in the United 
States: Core courses, concentration areas, and the ranking of the school. Strategic Management Journal. 20. 549-565. 

University of Georgia (1999). Exploring the future of adult education graduate programs. Ph. D. seminar 
group research proposal. Athens, GA: College of Education. 

Wharton (1991). The restructured Wharton MBA: Inventing a new paradigm. Almanac. The Wharton 
School, April 2. 



20-1 



ERIC 




Assessment of a Graduate Program in Human Resource Development: Perceptions of Key 
Stakeholders 



Julie A. Furst-Bowe 
Joseph Benkowski 
University of Wisconsin-Stout 

In 1995, a new master's degree program in training and HRD was implemented at a mid-sized 
public university. When the program had been offered for approximately four years, a systematic 
assessment was conducted to obtain feedback from multiple sources and to determine whether the 
program was meeting the needs of key stakeholders. Identified key stakeholders for this program 
included the program director, current students, program graduates, faculty members, and 
members of the program’s advisory committee 
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In 1995, a mid-sized public state University located in the Midwestern United States was authorized by the state 
university system to offer a Master of Science degree program in Training and Development. This program was 
developed in response to a need for training and human resource development professionals in the geographic area 
served by the University and an interest expressed by students and alumni who had completed undergraduate 
courses in this area. 

Enrollment in this new program has steadily increased from 35 students in the fall of 1995 to 84 students in 
the spring of 1999. As of May 1999, 76 students had graduated from this program. In addition to campus-based 
offerings, courses for the program are now offered on weekends and via distance learning technologies, including 
video-based courses and web-based courses, to multiple sites within the state. 

In the fall of 1999, when the program had been offered for approximately four years, a systematic 
assessment of the program was conducted to obtain feedback on the program from multiple sources and to determine 
whether the program was meeting the needs of key stakeholders. Identified key stakeholders for this program 
included the program director, current students, program graduates, faculty members, and members of the program’s 
advisory committee. 

Theoretical Framework 

In recent years, multi-source or multi-rater assessment methods have evolved from an innovative technique used 
only by senior managers in select organizations to an essential management tool used in a wide variety of 
organizational settings (Church and Bracken, 1997). An estimated 90 percent of Fortune 1000 firms use some form 
of multi-source assessment to evaluate employees, products, services and programs (Atwater & Waldman, 1998). 

In higher education, program evaluation often continues to be conducted with obsolete methods and models 
which impede comprehensive improvement efforts (Jasperro, 1998). According to the Education Criteria for 
Performance Excellence set forth by the Malcolm Baldrige National Quality Award Program (1999), educational 
programs and offerings should be continually evaluated and improved using multi-source evaluation techniques. 
This evaluation process should include information from students, faculty, employers and other key stakeholders; 
comparative data on other similar programs offered by other institutions; and use of assessment results. 

Using multi-source assessment techniques may be particularly appropriate for assessing an academic HRD 
program. Collecting data from multiple sources allows for the collection of richer data and allows cross-validation 
of findings from different groups (Willis & Kahnweiler, 1995). Sources consulted in the evaluation should be those 
individuals or groups that are in the best position to provide accurate .feedback on the program (Atwater and 
Waldman, 1998). These individuals or groups that have been identified as key stakeholders need to be informed of 
the evaluation process and how their input will be utilized and integrated into the process (Martin, 1998). For 
educational programs, key stakeholders may include potential students, current students, former students, parents, 
instructors, administrators, employers and funders (Mirabella & Wish, 2000). 
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Statement of the Problem 



Reports on the creation and evolution of academic degree programs in HRD are largely missing from the research 
literature (Willis and Kahnweiler, 1995). Little information was available on the quality and effectiveness of the 
Master of Science degree program in Training and Development at a mid-sized public university. The purpose of 
the study was to assess the quality and effectiveness of the program as perceived by key stakeholder groups. This 
study sought to answer the following questions: 

1. What are the strengths and weaknesses of the M.S. degree in Training and Development as perceived 
by current students? 

2. What are the strengths and weaknesses of the M.S. degree in Training and Development as perceived 
by graduates of the program? 

3. What are the strengths and weaknesses of the M.S. degree in Training and Development as perceived 
by faculty members associated with the program? 

4. What are the strengths and weaknesses of the M.S. degree in Training and Development as perceived 
by advisory committee members? 

5. What recommendations from the above stakeholder groups can be implemented to improve the 
program? 

Methodology 

The process used to collect data for this study included a self-study report generated by the current program director. 
This self-study report included comparative data from other universities with master’s level programs in human 
resource development. In addition to the self study, surveys were distributed to four key stakeholder groups: 
students, graduates, faculty, and advisory committee members to collect information of their attitudes and 
perspectives (Mirabella and Wish, 2000). The program director’s self-study and survey results were then reviewed 
by the researcher and the institution’s research coordinator and a report containing several recommendations was 
generated. 

The self study report included seven major sections (Planning and Review Committee, 1999). The first 
section included information on how the program relates to the University’s mission and goals. The second section 
included a description of the general and specific objectives of the program, the design of the curriculum, the 
teaching and learning methods used in the program and the nature of the student research requirement. The third 
section included a list of the primary instructors involved the program and their associated areas of knowledge, skills 
and expertise. 

The fourth section of the self study contained an explanation of the process for assessing student outcomes 
and how information resulting from student assessments is incorporated into the program. The fifth section included 
a description of the facilities; library, information and technology resources utilized by the program, and the sixth 
section included information on the demands for graduates and anticipated trends in the nature of positions in the 
field. The report concluded with a listing of the top five similar programs in the United States and included 
comparative information on each of these programs. 

In addition to the self study, primary data were collected from four key stakeholder groups: current 

students, program graduates, faculty members, and advisory committee members. The survey instruments used to 
collect data from each of these groups were developed by the campus Program Planning and Review Committee for 
the purpose of program assessment. These instruments have been used to review a number of graduate programs at 
the University. 

The faculty member and advisory committee member surveys were administered by mail to the various 
stakeholder groups. The student survey was distributed in classes by the researcher and the institution’s research 
coordinator. The alumni survey was administered by the University’s Office of Budget, Planning and Analysis as 
part of a university-wide graduate follow-up study. Four of the surveys were administered during the fall of 1999. 
However, the alumni survey was conducted in the spring of 1998. The results of the study were analyzed by the 
researcher, with the assistance of a program assistant. 

Results 

The results of the study can be divided into five sections: results of the self-study, results of the student surveys, 
results of the faculty surveys, results of the graduate surveys and results of the advisory committee surveys. A 
summary of the results by section is presented below. 
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Self Study Results 



The self study included a description of the program, including the objectives of the program, the 
curriculum, the research requirements and methods used to assess student outcomes. In the self-study, the program 
director had identified five high quality graduate programs in training and development. Comparative data were 
provided for each of these programs in relationship to the program being evaluated. 

Strengths of the program, as identified by the program director, include student enrollment, experienced 
faculty, and offerings to off campus and distance education students. Weaknesses of the program, as identified by 
the program director, include the need for additional faculty expertise and additional coursework in areas such as 
emerging technologies, organizational change and global HRD issues. The program director also expressed concern 
regarding the outdated classroom facilities and equipment. There was also concern regarding the lack of on-line 
support services for off campus and distance education students. 

Student Survey Results 

The student survey consisted of three sections: a section that included 9 demographic items, a section that 
included 18 statements for students to rate on a 5-point Likert-type scale and a third section that included 5 open- 
ended questions regarding the strengths and weaknesses of the program. Surveys were administered in classes by 
the researcher and the institution’s research coordinator. Surveys were also mailed to off campus students. A total 
of 73 students completed the surveys. Results of the survey are as follows: 

Table 1 . Student Survey Results 



The objectives of this program are clearly communicated to students 3.99 

Courses in this program achieve their stated objectives 4.05 

Assessment procedures appropriately measure student learning 3.92 

There is unnecessary overlap or duplication in courses in this program 2.75 

Students can complete this program in a reasonable time frame 4.30 

Faculty in this program are accessible to for advisement 3.97 

Faculty in this program are available outside of class 4.00 

Faculty in this program provide students with current information 4.22 

Program has enhanced my oral communication skills 4.01 

Program has enhanced my written communication skills 3.86 

Program has enhanced my problem-solving skills 3.96 

Program has enhanced my appreciation of diversity 3.15 

Program has prepared me to be successful in HRD 3.83 

Overall, this is a quality program 4.16 

If 1 had to do it again, 1 would choose this program 4.08 

Library resources are adequate for this program 3.79 

Technology resources are adequate for this program 3.45 

Classroom facilities are adequate for this program 3.66 



The strengths that were most commonly listed by students included knowledgeable and experienced faculty 
members, the accessibility of instructors, the applied nature of the courses, and the variety of ways in which the 
courses were delivered (evenings, weekends, internet-based and via interactive television). The weaknesses that 
were most commonly listed by students included a lack of entry-level employment opportunities for program 
graduates, duplication of content among courses, insufficient content on emerging technologies and inadequate 
administrative and support services for off campus and distance education students. 
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Alumni Surveys 

The alumni survey was divided into five areas: improved competencies, satisfaction with education, 
satisfaction with employment, overall satisfaction with program and an open-ended question regarding suggested 
improvements. A total of nine students who graduated from the Training and Development program were surveyed 
and five students responded to the survey. At the time of this study, spring 1998, only a small number of students 
had graduated from the program and current addresses were not available for many students. The small number of 
respondents makes it difficult to identify specific strengths and weaknesses and does not allow for the results to be 
generalized among all program graduates. 

In the first section, respondents were to rate their level of competency improvement in 13 competency areas 
on a scale of one to five. In 12 of the areas, speaking, writing, problem solving, organizing information, analyzing 
information, providing leadership, making decisions, conducting research, working in teams, thinking creatively, 
understanding other cultures and using computers, students rated themselves 3.5 or higher. Only one area, using 
math and statistics, received a lower rating (2.8). Eighty percent of the respondents were either satisfied or very 
satisfied with the overall quality of instruction and the availability of faculty members. This same percentage rated 
the value of the education they received as very good or exceptional. However, 40 percent (two of the five) 
respondents indicated they would not enroll in this same program again. 

Sixty percent of the alumni surveyed indicated that they had received “good” or “very good” preparation 
for employment and for career development; and this same number were satisfied or very satisfied with their rate of 
professional advancement. One hundred percent of the respondents were employed; however only one graduate 
reported that that he was working directly in the training and development field. The average salary of the graduates 
was $30,815. Recommendations from alumni included updating curriculum, requiring currency and consistency 
among instructors, increasing job placement efforts and offering more computer-based courses. 

Faculty Survey Results 

Six faculty members were identified as “primary instructors” in the program by the program director. These 
individuals teach required courses in the program and serve as advisors for student research papers. The faculty 
survey consisted of two sections: the first section included 16 statements regarding the program that faculty was 
asked to rate on a five-point Likert-type scale and the second section contained a series of open-ended questions 
regarding the program. The results of the faculty survey are presented below: 

Table 2. Faculty Survey Results 



Quality of instruction in required courses 3.40 

Relevance of information provided in required courses 3.25 

Quality of instruction provided in support courses 3.20 

Quality of scholarly activity/research by instructors 2.80 

Quality of classroom facilities 2.20 

Quality of laboratory facilities/technology resources 3.33 

Level of clerical support 3.20 

Students entering program are adequately prepared for graduate work 3.20 

Students graduating from program have mastered required content 3.20 

Students have the potential to be successful upon graduation 3.20 

Communication between faculty and the program director 3.80 

Leadership of the program director 3.80 

Cooperation of the department chair 4.00 

Library resources are adequate for student needs 3.40 

Library resources are adequate for faculty research needs 3.40 
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According to the faculty members, strengths of the program include course quality and strong leadership of 
the program director. Weaknesses of the program included inadequate number of faculty members to serve off 
campus and distance education students, lack of entry-level jobs for graduates, quality of the student research 
papers, content overlap among some of the required courses and variation in how research courses are taught. 

Advisory Committee Survey Results 

The program’s advisory committee consisted of five faculty members, one administrator, one student and 
six human resource development professionals. The average length of time a member had served on the program 
committee was 3.6 years. Each advisory committee member was asked to complete a questionnaire that included a 
series of open-ended questions regarding the strengths and weaknesses of the program. 

The major strengths of the program, as identified by the advisory committee members, included “hands-on 
experience” provided through internships, the program’s focus on performance, the program’s applicability to the 
corporate world and the dedication and experience levels of the faculty involved in the program. The weaknesses 
identified by the program committee included turnover in key faculty positions (including program director), 
insufficient number of faculty members to continue offering the program at multiple locations and the need for 
greater emphasis on web-based instructional design. 

Conclusions and Recommendations 

The following conclusions have been drawn based on the findings from this study. Recommendations based on 
these conclusions are as follows: 

Data from multiple sources indicate that there is overlap among the courses in the program. It is 
recommended that the program director, with the assistance of faculty members and the advisory committee, review 
the courses in the program and attempt to eliminate unnecessary duplication of content among courses. It is also 
recommended that the curriculum be examined in light of current trends in the field to ensure courses continue to 
include relevant content. 

Over the past five years, the program has expanded rapidly from a small, campus-based program to a 
program that serves students throughout the state. Although the courses have been made available to students 
through a variety of delivery methods, the number of faculty members has not increased. It also appears as though 
the campus is having difficulty providing administrative and student services to these students. It is recommended 
that additional staffing, both permanent and adjunct, be added to better serve the number of students in the program. 
It is also recommended that the program director work with the offices that provide administrative and support 
services to students to make these offices aware of the unique needs of students studying at a distance. 

Of major concern to the program director and faculty members were the lack of quality classrooms and 
instructional facilities for the program. It is recommended that the program director and department chair work with 
the campus planner to discuss possibilities for locating or developing better facilities for the program. 

The faculty members were also concerned about the quality and level of research and scholarship among 
themselves and their students. It is recommended that more emphasis by placed on the value of research by faculty 
and administrators. If additional staff members can be hired, the existing faculty members may have a greater 
amount of time to work on collaborative research projects with students. 

Data from students, alumni and faculty members indicate that it is difficult for students to obtain entry-level 
positions in the field of human resource development. In order to increase the number of students placed in 
positions related to their field of study, several recommendations were made including increasing the number of 
students who complete internships. Internships are critical for students who lack previous work experience in HRD 
(Schwindt, 1995). The program director, faculty members and advisory committee members are encouraged to 
increase networking efforts with organizations and to work the university’s placement office in identifying job 
opportunities for graduates. 

Finally it is recommended that multi-source assessment be conducted on a regular basis to ensure the 
program is meeting the needs and expectations of key stakeholders. It is recommended that additional input from 
program graduates be sought and that employers be added to the list of key stakeholders. It is also recommended 
that interviews with alumni, employers and advisory committee members be utilized in addition to surveys. 
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Knowledge Contribution to HRD 

Educators are increasingly faced with an every-growing body of literature focused on the strengths and weaknesses 
of diverse program evaluation approaches (Schnoes, 2000). This study provides a framework for an assessment 
model that can be applied to other academic programs in Human Resource Development. Identifying key 
stakeholders and developing systematic methods of collecting and analyzing input from these individuals and groups 
is essential for program improvement. Ratings from various groups provide different, relatively unique perspectives 
and these perspectives, when taken together, are more valid than that of a single evaluator (Borman, 1997). 
Programs are strengthened as stakeholder expectations are included in defining the direction of the program (Freed, 
1997; Obrecht, 1999). 

Although all of the data presented in this study were collected from one academic program, findings from 
this study could provide information for universities desiring to implement or expand an academic program in 
Human Resource Development (Henschke, 1995). The data from this study may also provide some important clues 
that may be considered by other university programs desiring to improve programs that prepare HRD professionals. 
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Master of Business Administration (MBA) program is one of the most popular approaches to 
management education. This study investigates the impacts of several precedent variables on the 
academic performance in an accredited MBA program. A prediction model was developed with 
multiple regression and the results showed significant impacts of undergraduate grade point 
average (GPA) and Graduate Management Admissions Test (GMAT ). Implications for 
management education are discussed. 
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Graduate study in Master of Business Administration (MBA) is one of the major approaches to management 
education (DeSimone, & Harris, 1998). The benefits of developing managerial skills through MBA program have 
been well documented. Sunoo (1999) comments that an MBA degree provides an opportunity for human resource 
professionals to enhance their competencies and boost their chances for career development. Messmer (1998) 
suggests that certified management accountants can benefit significantly from an MBA that offers the growing need 
for expertise in areas beyond the accounting department. Perry (1995) observes that many food scientists cannot 
advance to management positions because they get mainly technical training and very little of management 
education. He advises food scientists to take up MBA degrees in reputable universities in order to advance 
professionally in the food industry toward executive and management positions. Recent technology development 
has put MBAs with technology concentration in big demand. Those techno-MBA graduates not only have excellent 
technology skills, but also understand the strategic, business application of technology. Computerworld (1999) 
surveyed 63 techno-MBA programs and found that graduates of the best techno-MBA programs normally received 
multiple job offers and landed positions that pay $80,000 to $100,000 per year or more and offer perks such as 
lucrative stock options. Although graduate study no longer guarantees prestige, the MBA degree seems to retain its 
glamorous reputation (Shelley, 1997). 

On the other hand, demand for admission to the top MBA programs has been particularly strong and the 
cost of this type of management education is high. During the academic year of 1996-1997, American universities 
award more than 96,000 master’s degrees in business management and administrative services and that figure 
accounted for nearly one quarter of all master’s degrees conferred (Morgan, 1999). The total cost for the top 
executive MBA program reaches $87, 500 (Industry Week, 1999). Admission to the top MBA programs b very 
competitive. In 1999, the acceptance rate to Stanford University’s MBA program was 7% with 6,606 applicants and 
the acceptance rate in Columbia University was 1 1% with 6,406 applicants (Business Week Online, 2000). Given the 
highly competitive nature of MBA admission, one cannot help asking the question if the criteria commonly used in 
the admission decisions are predictable for the success in graduate management education. 

Similarly, management educators and administrators may also want to understand the factors that determine 
MBA students’ academic performance. On one hand, a good understanding of the factors influence students’ 
academic performance will help related parties to design appropriate academic program and supporting activities. On 
the other hand, a good knowledge of MBA students’ academic performance and its relation with major precedent 
variables will enhance the decision making in admission process. 

The purpose of this study was two fold: (1) to investigate the major precedent variables that significantly 
influence MBA students’ academic performance; and (2) to determine the extent to which a group of precedent 
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variables can successfully predict MBA students’ academic performance. 

Theoretical Framework 

The theoretical framework guided this study was based on evaluation model of human resource development 
interventions. Specifically, Holton’s (1996) conceptual model of evaluation outlines factors that determine individual 
performance and organizational results. Based upon existing evaluation models and research, Holton (1996) 
proposes causal relationships among motivation elements, environmental elements, ability/enabling elements, and 
outcomes. It is posited that individual performance is a function of learning outcome which, in turn, is influenced by 
motivation to learn and individual ability. In the context of management education, academic performance can be 
viewed as an immediate learning outcome and thus can be predicted by several precedent variables such as prior 
academic performance, motivation, and ability to learn. 

Related Literature 

Most graduate schools of management use some type of formula score that combines undergraduate grade point 
average (GPA), Graduate Management Admission Test (GMAT) and other quantifiable factors for admission to MBA 
(Carver & King, 1994). Underlying such common practice is the assumption that MBA students’ academic 
performance can be well explained by the precedent variables such as undergraduate academic performance and 
standardized test score (e.g., GMAT and MAT). Consequently, there has been always a concern whether such 
practice is theoretically justifiable and empirically valid (Carver & King, 1994; Schwan, 1988). A literature review 
suggests that there are a lot of studies devoted to investigated the relationship between MBA students’ academic 
performance (usually defined and measured by GPA) with some precedent variables. However, the literature on the 
prediction of academic performance in graduate management education is not conclusive and the empirical evidences 
are actually quite different. Hecht and Power (1982) report that the multiple correlation of undergraduate GPA and 
GMAT scores with first year MBA grade ranged from .12 to .67. Wright and Palmer (1994) used a sample of 86 MBA 
students at a small midwestem university to determine if GMAT scores and undergraduate GPA were better 
predictors of graduate performance for some groups of students than for others. It was hypothesized that these 
precedent measures were adequate predictors of low graduate performance. The results indicated that, although 
undergraduate GPA and GMAT scores were modestly associated with graduate performance across the full range of 
students, they did not discriminate between moderately low and very lower performers in the program. Multiple R- 
square was estimated as .212. 

The explanatory and predictive power of the certain precedent variables commonly used in graduate 
admission practice has been studied, but different results were yielded. Carver and King (1994) investigated the 
MBA admission criteria for nontraditional students. The researchers explored a number of precedent variables 
including: age, gender, undergraduate major, work experience, lapse of formal education, competitiveness of 
undergraduate institution, undergraduate GPA, and GMAT verbal (GMATV) and GMAT quantitative (GMATQ) 
scores. Nevertheless, they found that only three variables were the best predictors of success for the nontraditional 
students: GMAT score, undergraduate GPA and work experience (R 2 = .220). Paolillo (1982) reported that 
undergraduate GPAs and GMAT scores explained slightly less than 17% of variance in graduate GPA. Likewise, 
Deckro and Woundenberg ( 1 977) reported that GMAT score and undergraduate GPA accounted for less than 1 5% of 
the variance in academic performance of graduate management education. Hancock (1999) confirmed previous 
finding that there is no gender difference in MBA academic performance, but also revealed that males achieved 
higher scores on GMAT. 

Although it has been recognized the both undergraduate GPA and GMAT scores are needed as key 
admission criteria, previous studies revealed mixed results in terms of the relative impacts of these two variables on 
the graduate academic performance. Zwick (1993) conducted a study involved 90 schools in the United States and 
Canada to investigate the validity of the GMAT for the prediction of grades in doctoral study in business and 
management. It was found that the prediction achieved using undergraduate GPA alone as a predictor tended to be 
more accurate than that obtained using GMATV and GMATQ together. Including all three predictors was more 
effective than using only undergraduate GPA. In a series of bivariate regression analyses for the data set collected 
from a southeast university, Ahmadi, Raiszadeh and Helms (1997) reported that undergraduate GPA accounted for 
more than 27% of the variability in graduate GPA and GMAT explained only 1 8% of the variability. In a recent study 
of predicting MBA academic performance, Hoefer and Gould (2000) revealed similar finding to that of Zwick and 
found that GMATV, GMATQ and undergraduate GPA were strong predictors. Moreover, GMAT scores had higher 
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correlation with the graduate GPA than undergraduate GPA. Nevertheless, Carver and King (1994) report that GMAT 
is stronger predictor than undergraduate GPA in predicting MBA academic performance (standardized regression 
coefficient was .354 and .256 respectively for these two predictors). 

Research Questions 

Because the literature on the predictive power of those variables commonly used in graduate management admission 
is not conclusive, this study was designed to answer the following research question: 

1. What is the extent to which the academic performance in graduate management program can be explained by 
certain precedent variables? 

2. What is the relative importance of a group of precedent variables in explaining and predicting the academic 
performance? 

Methods 

Sample and Data Collection 

Files of all MBA graduates at Auburn University were sought and a total number of 543 subjects served the 
sample of this study. The MBA program is accredited by the AACSB. Among 543 subjects, 148 of them had missing 
information on at least one variable (27.3%) and thus were not entered into the data analysis. Consequently, the 
remaining 395 subjects were used as the valid sample of this investigation. 

Variable Selection 

It has been a common practice to use Grade Average Point (GPA) as an indicator of students’ academic 
performance. Following this tradition, MBA students’ academic performance was measured on the overall GPA (on a 
4.0 scale) in the present study. Therefore, MBA students’ overall GPA was treated as dependent variable. Several 
precedent factors were included to determine their influence on the dependent variable and these factors are 
generally terms as independent variables or predictors. Students’ gender was included in the study to see if sex have 
significant impact on students’ academic performance. There might a gender gap on standardized test such as 
GMAT and academic performance (Hirschelf, 1995; Johnson & McLaughlin, 1993). Students’ native language was 
another independent variable included in the study. It was reasoned that international students might have language 
barriers that ultimately affect their academic performance. Previous studies have shown that foreign students’ 
English fluency and country of origin affect academic performance (Stolzenberg & Rdles, 1991). Students’ 
undergraduate GPA was included in the study as it was reasoned that prior academic performance might hold 
continuos impact on the academic performance at graduate level. Finally, GMAT total score (GMAT), GMATQ and 
GMATV are used in this project because they are normally used as the important admission criteria in most graduate 
management education programs. The GMAT is designed to measure general ability and knowledge of the student. 

Data Analysis 

Multiple regression analysis is the analysis technique used in the study. This technique examines the 
multiple correlation between a dependent variable and a set of independent variables. Several analysis stages were 
involved in the data analysis in order to build a robust prediction model and to determine the generalizability of the 
model (Stevens, 1996). First, the whole sample was randomly splitted into two approximately equal number groups. 
One group served as model building sample used to establish a prediction model for MBA’s academic performance. 
The other group served as holding sample used to validate the model established for the model building sample. 
Second, a series of multiple regression analyses were conducted for the model building sample to establish a 
prediction model with best predictors. The regression assumptions were examined to see if they have been met in 
order to conduct the analysis. Thirds, the regression model established in the second stage was validated for the 
holding sample and the final model was built when validation evidence was sought. Fourth, the final model was 
applied to the whole sample to estimate relevant parameters in the model. All data analyses were conducted with 
JMP IN software program that was developed by SAS Institute (Sail & Lehman, 1996). 
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Results 



Table 1 presents the demographic information for each of the samples. Both male and female MBAs are almost 
equally represented in the building and holding samples. Female MBAs represent about one quarter of the student 
body. Only about 5% of the sample came from foreign countries. Furthermore, international students are not quite 
equally represented in the building and holding samples (3.96% and 5.71% respectively). 



Table 1. Demographic Distributions Across Samples 





Building Sample 


Holding Sample 


Total Sample 




N 


% 


N 


% 


N 


% 


Gender 














Male 


149 


73.76% 


143 


74.10% 


292 


73.92% 


Female 


53 


26.24% 


50 


25.90% 


103 


26.08% 


Language 














English 


194 


96.04% 


182 


94.30% 


376 


95.19% 


Foreign 


8 


3.%% 


11 


5.70% 


19 


4.81% 


Table 2 reports means 


and standard deviations of continuous variables across different samples. As it can 


be seen, all these continuous variables have by and large same means and SDs between the building and the holding 
samples. For the total sample, GMAT score ranged from 340 to 770, GMAT verbal scores ranged from 1 1 to 65, and 


GMAT quantitative scores ranged from 14 to 49. 


GPA scores 


in the MBA program 


ranged from 2.43 to 4.0 and the 


undergraduate GPA ranged from 2.1 3 to 4.0. 










Table 2. 


Means and Standard Deviations for the Variables in the Study 








Building Sample 


Holding Sample 


Total Sample 




Mean 


SD 


Mean 


SD 


Mean 


SD 


GPA@MBA 


3.43 


.30 


3.42 


.30 


3.42 


.30 


GMAT 


524 


77 


521 


73 


523 


75 


GMATV 


31.00 


6.77 


30.72 


6.49 


30.86 


6.63 


GMATQ 


31.20 


6.91 


30.91 


6.20 


31.06 


6.56 


UGPA 


3.04 


.40 


3.11 


.42 


3.08 


.41 



All predictors were entered into regression analyses for the building sample first. Different combinations of 
predictors were examined to find the best multiple correlation with graduate GPA. It was found that students’ age 
and gender had no significant predictability for the academic performance. Language showed somewhat significant 
prediction, while GMATQ, GMATV, and UGPA presented very strong prediction power. Then a final model was 
built based on predictors of language, GMATQ, GMATV, and undergraduate GPA. The model explains nearly 25% 
variation of the academic performance for the building sample. This model was then applied into holding sample and 
it accounted for about 26% variation of MBAs’ GPA. All regression estimates from the two samples were then 
compared and they are very close. The cross-validated R (between the observed GPA scores for the holding sample 
and the predicted ones based on the model developed from the building sample) was .51 (p < .001). It was then 
concluded that the model built for the building sample was reasonably applicable to the holding sample. 

Based on the above results, the regression model was used for the whole sample to get the regression 
estimates. Table 3 reports estimated parameters and associated statistical tests. The results of this study suggested 
that language was somewhat significantly predictable for the MBA students’ academic performance. GMAT 
Quantitative and Verbal, and undergraduate GPA were very significant in the prediction. R-square of the regression 
is .26. That is, more than one quarter variation of the MBA academic performance can be explained by the regression 
model. 

The relevant Ts, p-values, and standardized regression coefficients in the Table 3 provide information about 
the relative importance of the predictors. The larger the T and standardized regression coefficient and the smaller the 
p-value, the more important the predictor is. This study suggested that undergraduate GPA was the most important 



20-3 



20 



predictor for the graduate academic performance, followed by GMATQ and GMATV, while language made little 
contribution. 

Table 3. Regression Equation Predicting MBAs’ Academic Performance 



Predictors 


Estimate (B) 


Standardized B 


SEB 


T 


p-value 


Intercept 


1.899 


0 


.135 


14.05 


<.0001 


LANGUAGE 


.056 


.081 


.032 


1.75 


.081 


GMATQ 


.011 


.252 


.002 


5.23 


<.0001 


GMATV 


.001 


.213 


.002 


4.33 


<.0001 


UGPA 


.267 


.331 


.033 


8.16 


<.0001 



R 2 = .259 

^ (adjusted) — *252 

F(4,390) = 34.008 (p < .0001) 



Conclusions and Limitations of the Study 

Overall, this study confirms the findings revealed in the literature. It is a positive result that about one quarter 
variation of MBAs’ academic performance can be explained by only four precedent variables. This result tends to 
support the utility of GMAT and undergraduate GPA. It appears that undergraduate performance is the most 
important prediction for the graduate academic performance. The study discovered that age and gender had no 
predictive utility in explaining the academic performance. Clearly, admission decision and any other selection 
process should not be based on age or gender. 

Nevertheless, the predictive utility of precedent variables commonly used in graduate management 
education is limited. The study calls for further research in this area. Particularly, it is important to investigate other 
important variables such as learning motivation and working experience. Only about one quarter of the variance of 
MBA’s academic performance could be accounted for by several selective variables. Several authors have correctly 
noted that there are far more important variables that determine the MBA’s academic performance than those ones 
used in common admission practice (Ahmadi, Raiszadeh & Helms, 1997; Baldwin, Bedell, & Johnson, 1997; Wright & 
Palmer, 1994). It would be inaccurate to assume that prior academic performance is the single best predictor for the 
performance in management education program. This study reveals that only one quarter of the variations of 
academic performance can be attributed to some precedent variables. Baldwin et al. (1997) find that centrality in 
friendship, communication, and adversarial network affect both MBA students attitudes and grades. Management 
educators should pay more attention to the learning contexts that determine learning and transfer. Admission 
decisions should be made incorporating other evidences such as writing samples, career statement, personal 
interview and reference. 

There are certainly a number of limitations of this study that constrain the generalizability of this study and 
warrant some caution in the interpretation of results. First, only few predictors were sought due to limited time and 
resources. Had other important variables been included in the project, the prediction might have been improved to a 
greater extent. Second, there is considerable amount of missing data and thus the valid sample size has been 
considerably reduced. So far, we do not have enough information to infer how these MBAs would perform in 
relation with their backgrounds. Third, few international students are included in the study. Thus, it should be 
careful to generalize the conclusion to a larger population with regard to the admission practice. 

Implications for Management Education 

DeSimone and Harris (1998) comment that management education is one of the most common human 
resource development activities. Keys and Wolfe (1988) define management education as “the acquisition of a broad 
range of conceptual knowledge and skills in formal classroom situations in degree-granting institutions” (p. 205). 
This study contributes to the literature by examining several precedent variables that significantly influence academic 
performance in popular MBA program. Although the predictive power for the academic performance in graduate 
management education from prior performance and standardized test scores is limited, several precedent variables 
still explained a considerable variations of the academic performance. Management educators and administrators 
should carefully examine the impacts of prior academic performance on the performance of graduate management 
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education. They should work with test developers to ensure adequate validity and reliability of the standardized 
examination such as GMAT. Overall, this study supports the continuous use of undergraduate GPA and GMAT 
score in admission decision-making process. However, other factors should be taken into account such as 
motivation to learn, working experience, and career plan. 

In addition to the information from standardized tests and prior academic performance, other screening 
methods should be employed in graduate management admission. Tarr (1 986) observed that business schools have 
allowed concern for human skills to slip in their effort to strengthen technical scholarship. He emphasizes leadership 
skills in screening applicants. Assessment on applicants 1 motivation to learn, communication and leadership skills 
can be obtained through personal interview or other authentic assessment methods. 

This study has implications for management education not only in the area of admission decision but also in 
areas of content and teaching methods. Just like a limited percentage of variance of academic performance is 
attributed to those tangible previous learning outcomes such as undergraduate GPA and GMAT scores, MBA 
graduates 1 management performance in practice and career success cannot be explained solely by their academic 
performance. O’Reilly and Chatman (1994) studied the effects of motivation and ability on the early career success 
of a sample of MBA graduates in the early years of their careers. It was found that it was the interaction of 
motivation and general cognitive ability that most strongly predicted early career success. Nevertheless, the 
predictive power of these two variables was limited in terms of R 2 in multiple regression. After controlling several 
demographic variables (e.g., age, sex, years of graduation) and working settings (e.g., types of working organization), 
motivation and general ability accounted for only 4% variance of reported salary and 16% variance of promotion. 
The present study appears to concur with this finding and suggests that formal learning is not the most importance 
determinant for individual performance. Recent arguments have suggested that learning from practice or informal 
learning is as vital as formal learning of formal technical knowledge (Lave & Wenger, 1991; Yang, 1999). Traditional 
management education emphasizes theory rather than practice, and it normally values formal rather informal learning. 
This type of management education places much emphasis on rational, scientific, systematic and formal knowledge 
base as its major content. Raelin (1993) posits that advanced management education programs should include both 
theory and practice. Theory-based programs might cause students to think that management problems can be 
nestled into neat technical packages. In the light of holistic perspective of knowledge and learning, practice should 
be an essential component of management education. 

This study demonstrates that business education needs to be enhanced with international perspective. The 
results of this study suggest that students 1 native language had moderate influence on the academic performance. 
The factor of students 1 native language should be taken into account in graduate management education. As the 
world economy is experiencing rapid process of globalization and American firms are facing the challenge of 
international competition, universities are receiving more and more international students. According to the statistics 
of National Center for Education Statistics, a total of 14,389 master’s degrees in the field of business management 
were conferred by American universities to “non-resident alien” and this figure accounted for almost 15% of the 
degrees awarded in this field. The presence of foreign students can be a very positive force to graduate management 
education because they can share different cultural and social understandings about management and international 
backgrounds. Kedia and Harveston (1998) posit that management education needs to change to prepare business 
leaders with a worldview. Obviously, international students can be a valuable asset for any program of graduate 
management education. They can help faculty members to enhance awareness of international implications and 
global perspectives. However, they might have language barrier to overcome. As international students who have 
studied business management in two American universities, our experience showed that few faculty members have 
paid special attention to international students. Most faculty members tend to treat international students the same 
as American students and fail to consider their special needs and use this type of valuable asset. If we think that the 
concept of globalization is real, educators of business management should take a proactive role in increasing 
international perspective and examining global implications of their business. 
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